Skip to content

[Text] Implement Unicode word segmentation and update to Unicode v17 - #21361

Merged
Gillibald merged 5 commits into
AvaloniaUI:masterfrom
JimBobSquarePants:unicode-word-break
May 18, 2026
Merged

[Text] Implement Unicode word segmentation and update to Unicode v17#21361
Gillibald merged 5 commits into
AvaloniaUI:masterfrom
JimBobSquarePants:unicode-word-break

Conversation

@JimBobSquarePants

Copy link
Copy Markdown
Contributor

What does the pull request do?

I thought I'd spread the .NET community love a little and supply some fixes for things I know.

This PR updates Avalonia’s Unicode data generation to Unicode 17 and regenerates the affected Unicode trie/source files.

It also adds Unicode 17 grapheme support for Indic conjunct breaks, using the existing Avalonia grapheme trie storage, and updates the grapheme enumerator for GB9c.

I aligned line breaking behavior with the Unicode 17 data and provides fixes, including LB9 folding, quotation handling, Hebrew hyphen behavior, Aksara rules, and emoji modifier cases.

Validation run locally in Release:

  • Avalonia.Base build passed
  • Avalonia.Base.UnitTests build passed
  • Line break conformance passed
  • Grapheme conformance passed
  • Word break conformance passed
  • BiDi class and algorithm conformance passed

I've tried to match existing style as much as possible and added inline comments where applicable. I was very tempted to simply replace your LineBreakEnumerator implementation with the one from SixLabors.Fonts but didn't want to add too many changes.

Checklist

Breaking changes

Obsoletions / Deprecations

Fixed issues

Fixes #16524

@Gillibald

Copy link
Copy Markdown
Contributor

I truly appreciate this. Thank you.

@cla-avalonia

cla-avalonia commented May 13, 2026

Copy link
Copy Markdown
Collaborator
  • All contributors have signed the CLA.

@JimBobSquarePants

Copy link
Copy Markdown
Contributor Author

@cla-avalonia agree

@avaloniaui-bot

Copy link
Copy Markdown

You can test this PR using the following package version. 12.1.999-cibuild0065353-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot

Copy link
Copy Markdown

You can test this PR using the following package version. 12.1.999-cibuild0065369-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot

Copy link
Copy Markdown

You can test this PR using the following package version. 12.1.999-cibuild0065405-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot

Copy link
Copy Markdown

You can test this PR using the following package version. 12.1.999-cibuild0065459-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@Gillibald Gillibald left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again for this contribution. LGTM

@Gillibald
Gillibald enabled auto-merge May 18, 2026 06:10
@Gillibald
Gillibald added this pull request to the merge queue May 18, 2026
Merged via the queue into AvaloniaUI:master with commit 6fa084a May 18, 2026
11 checks passed
@JimBobSquarePants
JimBobSquarePants deleted the unicode-word-break branch June 17, 2026 12:21
@MrJul MrJul added the backport-candidate-12.0.x Consider this PR for backporting to 12.0 branch label Jun 23, 2026
MrJul pushed a commit to MrJul/Avalonia that referenced this pull request Jun 23, 2026
…valoniaUI#21361)

* Add word break support and use it for ellipsis

* Regenerate Unicode text-formatting data to v17 and fix issues

---------

Co-authored-by: Jumar Macato <16554748+jmacato@users.noreply.github.com>
Co-authored-by: Benedikt Stebner <Gillibald@users.noreply.github.com>
@MrJul MrJul added backported-12.0.x and removed backport-candidate-12.0.x Consider this PR for backporting to 12.0 branch labels Jun 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Text] Implement Unicode word segmentation

6 participants